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Abstract — We study the limits of communication efficiency 
for function computation in collocated networks within the 
framework of multi-terminal block source coding theory. With 
the goal of computing a desired function of sources at a sink, 
nodes interact with each other through a sequence of error-free, 
network-wide broadcasts of finite-rate messages. For any function 
of independent sources, we derive a computable characterization 
of the set of all feasible message coding rates - the rate region - 
in terms of single-letter information measures. We show that 
when computing symmetric functions of binary sources, the 
sink will inevitably learn certain additional information which 
is not demanded in computing the function. This conceptual 
understanding leads to new improved bounds for the minimum 
sum-rate. The new bounds are shown to be orderwise better 
than those based on cut-sets as the network scales. The scaling 
law of the minimum sum-rate is explored for different classes of 
symmetric functions and source parameters. 

I. Introduction 

Both wired and wireless data networks such as the Internet 
and the mobile ad hoc and wireless mesh networks have been 
designed with the goal of efficient data transfer as opposed 
to data processing. As a result, computation takes place only 
after all the relevant data is moved. Two-way interaction 
would be utilized to primarily improve the reliability of data- 
reproduction than data processing efficiency. However, to max- 
imize the data processing efficiency, it may be necessary for 
nodes to interact bidirectionally in multiple rounds to perform 
distributed computations in the network. In this paper we 
attempt to formalize this intuition through a disUibuted func- 
tion computation problem where data processing efficiency is 
measured in terms of the total number of bits exchanged per 
sample computed. Our objective is to study the fundamental 
limits of multiround function computation efficiency within a 
distributed source coding framework, involving block-coding 
asymptotics and vanishing probability of function-computation 
error, for "collocated" networks where broadcasted messages 
can be heard by all nodes. We derive an information-theoretic 
characterization of the set of feasible coding-rates and explore 
the benefit of multiround communication. 
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of R Gupta was supported in part by NSF Grant CNS-05 19535. Any opinions, 
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those of the authors and do not necessarily reflect the views of the NSF. 



This problem was studied in [1] within a communication 
complexity framework where computation is required to be 
error-free. For collocated networks and random planar mul- 
tihop networks, the scaling law of the maximum rate of 
computation with respect to a growing size of the network 
was derived for divisible functions and two subclasses of 
symmetric functions namely type-sensitive and type-threshold 
functions. This work was extended in [2] to multihop networks 
having a finite maximum degree. In [2] it was also shown that 
for any network, if a nonzero per-sample error probability was 
allowed, the computation of a type-sensitive function could be 
converted to that of a type-threshold function. In [3] a min-cut 
bound was developed for acyclic network topology and was 
shown to be tight for tree networks and divisible functions. 

In [4], a function computation problem in a collocated 
network was posed within a distributed block source cod- 
ing framework, under the assumption that conditioned on 
the desired function, the observations of source nodes are 
independent. An information-theoretic lower bound for the 
sum-rate-distortion function was derived. It was shown that if 
the desired function and the observation noises are Gaussian, 
the lower bound is tight and there is no advantage to be 
gained, in terms of sum-rate, by broadcasting messages, in 
comparison to sending messages through point-to-point links 
from source nodes to the sink where the function is desired to 
be computed. Multiround (interactive) function computation 
in a two-terminal network was studied in [5], [6] within a 
distributed block source coding framework. 

The impact of transmission noise on function computation 
was considered in [7]-[9] but without a block coding rate, 
i.e., only one source sample is available at each node. A joint 
source-channel function computation problem over noninter- 
active multiple-access channels was studied in [10]. Our focus 
is on the block source coding aspects of function computation 
and we assume that message exchanges are error-free. 

The present work studies a multiround function computa- 
tion problem in a collocated network within a multi-terminal 
source coding framework described in Sec. [II] Sensors observe 
discrete memoryless stationary sources taking values in finite 
alphabets. The goal is to compute a samplewise function at a 
sink with a probability which tends to one as the block-length 
tends to infinity. We derive a computable characterization of 
the rate region and the minimum sum-rate in terms of in- 



formation quantities (Sec. [TlTt. For computation of symmetric 
functions of binary sources, the sink is shown to inevitably 
obtain certain additional information, which is not demanded 
in computing the function (Sec. II V- Al l. This key observation 
is formalized under the vanishing block-error probability cri- 
terion (Lemma |2) and also the zero-error criterion (Lemma [3]). 
This conceptual understanding leads to improved bounds for 
the minimum sum-rate (Sec. IIV-Bb . These bounds are shown 
to be orderwise better than cut-set bounds as the size of the 
network grows. The scaling law of the minimum sum-rate is 
evaluated in different cases in Sec. IIV-CI 

IT. Multiround Computation in Collocated Networks 

Consider a network consisting of m source nodes numbered 
l,...,m, and one (un-numbered) sink (node). Each source 
node observes a discrete memoryless stationary source taking 
values in a finite alphabet. The sink has no source samples. For 
each ye [l,m]01et X ; - := (X/l), ... ,Xj(n)) e (Xj) n denote the 
n source samples which are available at node- j. To isolate the 
impact of the structure of the desired function on the efficiency 
of computation, we assume sources are independent, i.e., for 
i=l,...,n, (Xi(0, X 2 (i), X m (i)) ~ iid p x » = n7=i Px r Let 
/ : K\ x. . .xX,„ — » Z. be the function of interest at the sink and 
let Z(/) := f(Xi(i), . . .,X m (i)). The tuple Z := (Z(l), . . .,Z(n)), 
which denotes n samples of the samplewise function of all the 
sources, is desired to be computed at the sink. 

The communication takes place over r rounds. In each 
round, source nodes broadcast messages according to the 
schedule 1 , . . . , m. Each message depends on the source sam- 
ples and all the previous messages which are available to the 
broadcasting node. Nodes are collocated, meaning that every 
broadcasted message is recovered without error at every node. 
After mr message broadcasts over r rounds, the sink computes 
the samplewise function based on all the messages. 

Definition 1: An r-round distributed block source code for 
function computation in a collocated network with parameters 
(r, n, \Mi\, . . . , \Mt\) is the tuple (ei, . . . , e,,g) of t := mr block 
encoding functions e\, . . . , e, and a block decoding functions g, 
of block-length n, where for every j e [1, t], k = (j mod m)Jj 

y-i t 
ej : QC k ) n x (g) M, ■ -> Mj, * : (g) At; -> Z. 

'=1 7=1 

The output of ej, denoted by Mj, is called the j-th message, r 
is the number of rounds, and t is the total number of messages. 
The output of g is denoted by Z. For each j, (l/«)log 2 \Mj\ 
is called the j-th block-coding rate (in bits per sample). 

Remarks: (i) Each message Mj could be a null message 
(\Mj\ = 1). By incorporating null messages, the multiround 
coding scheme described above subsumes all orders of mes- 
sages transfers from m source nodes, and an r-round coding 
scheme subsumes an r'-round coding scheme if r' < r. (ii) 
Since the information available to the sink is also available to 

-When a and b are integers, [a, b] denotes an integer interval, which is the 
set of all consecutive integers beginning with a and ending with b. 
3 k = (j mod m) means that k e [l,m] and in divides (k - j). 



all source nodes, there is no advantage in terms of sum-rate 
to allow the sink to send any message. 

Definition 2: A rate tuple R = (Ri, . . .,R,) is admissible for 
r-round function computation if, Ve > 0, 3 h{e, t) such that 
Vn > n(e, t), there exists an r-round distributed block source 
code with parameters (r, n, \M\\, . . . , |A1 r |) satisfying 

V;e[l,f], - log 2 \Mj\ < Rj + e, P(Z + Z) < e. 

The set of all admissible rate tuples, denoted by is called 
the operational rate region for r-round function computation. 
The minimum sum-rate R sum ,r is given by minR e ft,. (2y=i Rj)- 
Note that since each message could be a null message, if r' < 
r, Rsumr ^ R.mm,r holds. The goal of this work is to obtain a 
single-letter characterization of the rate region (a computable 
characterization independent of block-length «), to study the 
scaling behavior of R sum ,n an d to investigate the benefit of 
multiround function computation. 

III. Rate Region 

The rate region for r-round function computation for m 
independent sources can be characterized by Theorem Q] in 
terms of single-letter mutual information quantities involving 
auxiliary random variables satisfying Markov chain and con- 
ditional entropy constraints. 

Theorem 1: 

K = {R | 3 U',s.t. V; e [l,f], and k = (j mod m), 

Rj > I(X k ; Uj\U j -\ Uj - (U j -\X k ) - {X k ~\X™ +l \ 
H(f(X m )\U') = 0}, (3.1) 

where U' are auxiliary random variables taking value in finite 
alphabets. Cardinality bounds on the alphabets of the auxiliary 
random variables can be derived using the Caratheodory 
theorem but are omitted. 

The proof of achievability follows from standard random 
conditional coding arguments and is briefly outlined as fol- 
lows. For the j'-th message, j = 1, . . ., t, node-A: (k = j mod m) 
quantizes X* into U; with U^ 1 as side information, which is 
available at every node, so that every node can reproduce V j. 
After all the message transfers, the sink produces Z based on 
U'. The constraints in (13. Il l ensure that P(Z = Z) — > 1 as 

77 — > OO. 

The (weak) converse, given in Appendix [A] is proved using 
standard information inequalities, suitably defining auxiliary 
random variables, and using time-sharing arguments. Specif- 
ically, U\ := (Q,Ui(Q)), Q ~ Uniform[l,n] independent of 
X'", for all q e [1,«], U x {q) = {Mi,Z m (l), . ..,X m (q- 1)}, and 
for all j e [2,f], Uj := Mj. 

By adding all the rate inequalities in ( 13. Il l and enforcing all 
the constraints, we have the following characterization of the 
minimum sum-rate. 

Corollary 1: 

Rsumr = mmI(X"';U'), (3.2) 

V 

where U' are subject to all the Markov chain and conditional 
entropy constraints in ( 13. 11 1. 



The Markov chain and conditional entropy constraints of 
( 13. Il l imply a key structural property which U* need to satisfy. 
This property is described below in Lemma Q] This lemma 
provides a bridge between certain fundamental concepts which 
have been studied in the communication complexity literature 
[11] and distributed source coding theory. In order to state 
the lemma, we need to introduce some terminology used 
in the communication complexity literature [11]. A subset 
3\ c (g). =1 Xi is called a rectangle if for every i e [l,m], 
there exists Si Q Xi such that M = &)"L\ Sj. A set J[ is 
called f -monochromatic if the function / is constant on M. 
The support-set of a probability mass function p is the set 
over which it is strictly positive and is denoted by supp(p). 

Lemma 1: Let U' be any set of auxiliary random vari- 
ables satisfying the Markov chain and conditional entropy 
constraints in ( I3.lt . If supp(px») = (g) /=1 Xi, then for any 
realization u> of U', ft(u') := {x m \p x ,»u<{x m , u>) > 0} is an /- 
monochromatic rectangle in <Y,\ 

Proof: The Markov chains in d3.lt induce the following 
factorization of the joint probability. 

Px>"U<(x"\ "') = Px'"{x m )pu i \x i {ui\ x \)Pu 1 \x 2 uM2\ x 2^ «i) ■ • ■ 

m 

=: px'-(x'")Y]Mxi,u'), 
i=i 

where <p, is the product of all the factors having conditioning 
on Xi. For each i e [1,/n], let Si(u') := {x; | 0,-(x,-, u') > 0}. 
Since Vx m , p P ,(f") > 0, we have Mu') = (g)™, S,<«0- Since 
#(/(X m )|t/') = holds, mill') is /-monochromatic. ■ 

IV. Computing Symmetric Functions of Binary Sources 

In this section, we focus on the problem of computing 
symmetric functions of m nontrivial Bernoulli sources: V« e 
[1,/n], Xi = {0, 1), px,0) = where 7r ; € (0, 1). Symmetric 
functions are invariant to any permutation of their arguments. 
A symmetric function f(X' n ) of binary sources is completely 
determined by the (integer) sum of the sources S := Z"i^Q- 
In other words, 3f : [0, m] -> Z, such that f'(s) = fix™). 

Definition 3: Given a function /' : [0, m] — > Z., an interval 
[a,b] c [0, m] is a maximal /'-monochromatic interval if (i) 
it is /'-monochromatic and (ii) it is not a proper subset of an 
/'-monochromatic interval. 

The collection of all the maximal /'-monochromatic in- 
tervals can be constructed as follows. First, consider all the 
inverse images {f'~ l (z)} z ez ■= ii s \ s e [0,m],/'O) = z}} z e z- 
Next, each inverse image can be written as a disjoint union 
of nonadjacent intervals. The collection of all such intervals 
from all inverse images, denoted by {[a v ,b v ]}^, forms the 
collection of all the maximal /'-monochromatic intervals. Note 
that they also form a partition of [0, m]. Without loss of 
generality, we assume that these intervals are ordered so that 
a\ = 0,b Vma = m and Vv e [2, v max \,a v = b v -\ + 1. 

A. Sink learns more than the result of function computation 

Note that if f(S) = z then S e / ,_1 (z) which is, in general, a 
disjoint union of several maximal /'-monochromatic intervals. 
Thus, if the sink can successfully compute the function f'(S), 



one may expect that the sink can only estimate the value of 
S as belonging to the union of several intervals. Somewhat 
surprisingly, however, it turns out that due to the structure of 
the multiround code, the sink will, in fact, be able to identify 
a single maximal monochromatic interval to which S belongs 
as opposed to the union of several intervals. More surprisingly, 
the sink will be able to correctly identify the source-values at 
certain nodes. Lemma [2] formalizes this unexpected property 
and plays a central role in proving Theorem |2ji). 

Lemma 2: Let f(x m ) be a symmetric function of binary 
variables and {[^v.fcv]}!^ the collection of all the maximal 
/'-monochromatic intervals associated with /. Let X m be 
m independent nontrivial Bernoulli random variables and U' 
auxiliary random variables which satisfy the Markov chain 
and conditional entropy constraints in d3.lt . Then for any 
u' e supp(/?;/<X the following conditions hold. 

(i) There exists v(u') e [l,v max ] such that 

P(S e [a v («o A(«oW' = "') = 1. 

(ii) There exist < K\(u t ) c [1,/n] and 7C (m') c [1,/n] such that: 
Ki(iO n W) - {}, IftiCiOl > a vm , ITCoOOl > m - b vW , and 

P(Vz e TCi(m'), V/' e %i(vt),Xi = l,X f = 0\U' = u') = 1. 

Proof: Due to Lemma Q] Jl(u') = {x m \px^v<{^,u t ) > 0} 
is an /-monochromatic rectangle, which can be expressed as 
0" ! Si(u'), where Si(u<) is either (0) or {lj or {0,1}. Let 
W) := [i | Si(u') = {1}} and <K (u t ) := {i | Si(u') = {0]}. 
Let a(u') := |^fi(w')l and P( u ') ■= m-\%3 («*)!■ It can be shown 
that the projection of J\{u') under the linear transformation s = 
x ) g^en by Sl'W) := ((Z^i ^) l^-»£/<(^, «0 > 0} is an 
/'-monochromatic interval [a{u'),f3(u')]. Since {[a v ,& v ]} v ™j I is 
the collection of all the maximal /'-monochromatic intervals, 
3 v(u') e [l,Vmax] sucn that J['{u r ) c [a v ( M <)>^v(«o]- Therefore 
(i) holds. Since Ji'(u') = [a(i/), /?(«')] c [«,,(„<), &,,(„')], we have 
a(w f ) > a„( M t) and /3{u') < b V ( U i). Therefore (ii) holds. ■ 
Although learning that f'(S) — z is equivalent to learning 
that S e f'~ l (z), which is generally a union of several intervals, 
Lemma |2i) shows that the structure of block source coding 
for function computation in collocated networks is such that 
the sink will inevitably learn the exact interval [a V ( M i), fc V (« f )] m 
which S resides even though this information is not demanded 
in computing f'(S). Similarly, Lemma HJii) shows that al- 
though learning that S 6 [a V ( U <), b v ( u t)] is equivalent to learning 
that there exist a l( y) nodes observing ones and {m-b V ( U i)) nodes 
observing zeros, the sink will inevitably learn the identities of 
these nodes. 

Lemma |2] describes a property of the single-letter char- 
acterization of the rate region. It does not, as such, have a 
direct operational significance. Hence the conclusions of the 
previous paragraph can be only accepted as intuitive inter- 
pretations. If, however, the block-error probability criterion 
P(Z + Z) < e in Definition |2] is replaced by the zero-error 
criterion P(Z + Z) = as in [1], we obtain Lemma [3] which 
holds for every sample realization and provides an operational 
significance to the results suggested by Lemma [2] 



Lemma 3: Let f(x m ) be a symmetric function of binary 
variables and {[fl v >^v]}v™ tne collection of all the maxi- 
mal /'-monochromatic intervals associated with /. Let X m 
be m independent nontrivial Bernoulli sources. For any r- 
round, block-length n code@ for computing / in a collo- 
cated network, if P(Z + Z) = 0, then given all the 
messages M', for every sample i e [l,n], the following 
conditions hold, (i) There exists v{M',i) e [l,v max ] such 
that S(i) := (zZ%i x j(i)) e [a v{ M',i),b v(M ',i)])- (ii) There ex- 
ist K\(M',i) c [l,m] and r K (M',i) c [l,m] such that: 

TdCM'.OnW^.O = {KITCiCM'.OI > flvCM'.o.lW'.OI > 
m-b mKt) , and Vy e <Ki(M',i),Vf e <K {M< ,i),Xj(i) = 
l,X r (i) = 0. 

The proof of Lemma [3] given in Appendix [B] is similar in 
structure to those of Lemmas Q] and [2] 

Example: (Parity function) Let f(x m ) — ((J)™ j XjJ be the 
Boolean XOR function (parity) of m binary variables. Then 
/'-i(0) = U, L !f J {2/} and /'-'(D = U[!f 1 {2/ - 1}. Thus for all 
v e [1, (m+l)], a v - b v = (v-1), and all the /'-monochromatic 
intervals are singletons. For every sample i e [l,ri], if / is 
computed with zero error, Lemma |3ji) shows that the sink 
ends up knowing S(i) exactly, because every interval is now 
a singleton. In addition, Lemma |3jii) shows that the sink will 
also identify S (/) source nodes which observe ones and (m - 
S(i)) source nodes which observe zeros. Therefore the sink 
essentially needs all the raw data from all the source nodes in 
order to compute the parity function in a collocated network. 

B. Bounds for minimum sum-rate 

Returning to the block-error probability criterion, Lemma [2] 
leads to the following bounds for R sumj - when X m ~ iid 
Bernoulli(p), that is, V/ e [l,m], 7r,- = p. 

Theorem 2: Let fix" 1 ) be a symmetric function of binary 
variables and {[a v ,fo v ]}^ the collection of all the maximal 
/'-monochromatic intervals associated with /. If X m ~ iid 
Bernoulli(p), p e (0, 1), then for all r e Z + , (i) 

Rs Um , r >mh(p)- 2 (b v -a v )h(^^^t±)F(S e[a v ,b v ]), 

v=\,a,±b v 

(ii) R mmr >mh(p) max [P(S < b v )¥(S > b v )], 

Z- I a v m — b v \ 
- + - -}nS €[a v ,b v ]), 
r=1 \ P 1 " P I 
where h{-) is the binary entropy function. 

Remark: The minimum sum-rate for "data downloading" 
where all source samples are to be reproduced at the sink is 
H(X"') = mh(p), Theorem|2jii) explicitly bounds the efficiency 
of multiround broadcasting relative to data downloading. Since 
(ii) is proved by relaxing the lower bound in (i), the right side 
of (ii) is not greater than that of (i). 

4 The results of Lemma [5] hold for not only the multiround block coding 
strategy described in Definition[T]but also for the class of collision-free coding 
strategies defined in [1]. 



C. Scaling law of minimum sum-rate 

Consider a sequence of problems, where in the m-th prob- 
lem, m € Z + , m source nodes observe Bernoulli(p m ) source 
samples X" 1 which are iid both across samples and across 
nodes and /„, is the desired function. Let R SU m,r,m be the 
minimum sum-rate of the m-th problem. The scaling law of 
Rsum,r,m with respect to m is explored in the following cases. 

Case 1: We need to use the following fact. 

Fact 1: For any e e (0, 1/2), if max„ e[ i,, w] [P(S < b v )¥(S > 
b v )] < e(l-e), then 3 v* e [l,v max ] such that P(5 e [a v ,,b v ,]) > 
1 -2e. 

Proof: For any e E (0,1/2), if max l , e[ i,„ miix] [P(5 < 
b,)V(S > b v )] < e(l-e), then Vv,P(5 < b v ) e [0, e) U(l -e, 1], 
which in turn implies that for v* = min{v|P(5' < b v ) > 1 — e}, 
P(5 e [a v .,M) = F(5 < b v *)-V(S < V-i) > l-2e holds. ■ 

If 3e e (0, 1 /2) such that for every maximal f' m - 
monochromatic interval [a v , m , &v,m], P(S 6 [a v , m ,b v , m ]) < (1 - 
2e), then due to Fact [Q max v£[ljVnM<] [P(5 < b v )F(S > b v )] > 
e(l — e). Then due to Theorem Hfii), R sllm ,r,m - e(l - e)mh(p m ), 
which implies that R SU m,rjn = ®(mh(p m )) and data downloading 
is orderwise optimal. Conversely, if R S um,r,m - o(mh(p m )), then 
due to TheoremHtii), max,, e[1 ,, Wx] [P(5 < b v )¥(S > b v )] -> as 
m — > oo. Therefore there exists a vanishing sequence {e m } me z+ 
such that max,, E[ i ilw] [P(S < b v )¥(S > b v )] < e m (l - e m ) holds 
for the m-th problem. Due to FactQ] there exists a sequence of 
maximal /^-monochromatic intervals {[flv",m, b v * >m ]} m€ z+ such 
that P(5 £ [« v *,m> ^v*,m]) ^ 1 as m —> oo. In other words, 
multiround computation of symmetric functions of iid binary 
sources in collocated networks is orderwise more efficient than 
data downloading only if each sample of f„ is determined with 
a probability which tends to one asm^ oo0 

Case 2: (p m = 1/2) For any symmetric function of iid 
Bernoulli(l/2) sources, let p := m - £ v ™[' (b v — a v ) P(5 e 
[a v ,b v ]). Theorem |2ji) and (iii) imply that p < R sumj < 
Rsum,\ - 2p. This shows that multiround computation can at 
most halve the minimum sum-rate of one-round computation. 
Since p can be easily computed using the binomial distribution, 
Rsum,r can be easily evaluated within a factor of 2 for all r e Z + . 

Case 3: (Type-sensitive functions, p„, — 1/2) A sequence 
of symmetric functions {/„} me z+ of binary variables is type- 
sensitive if 3 y e (0, 1) and m e Z + such that Vm > in, for 
every f' m -monochromatic interval \a vjm , b l%m ], (b v m -a vjn ) < yfh 
(defined in [1], adapted to our notation). For example, the 
sum, mode, and parity functions are type-sensitive. For iid 
Bernoulli(l/2) sources, it can be shown that R mm , r , m - ®{m) by 
applying Theorem Remark: For the zero-error criterion, 
the minimum worst-case sum-rate is also 0(m) [1]. 

Case 4: (Type-threshold functions) A sequence of symmetric 
functions {f m } me z+ of binary variables is type-threshold, if 
there exist 6q, 6\ 6 N such that [9\ , m-f?o] is f' m -monochromatic 
for every m >9o + 6\ (defined in [1], adapted to our notation). 

5 We cannot, however, let nodes send nothing and set the output of the 
sink to be the determined function value because then, for each m the 
probability of block error will tend to one with increasing block-length 
violating Definition [2] 



For example, the minimum and maximum functions are type- 
threshold, (i) If p m = p, then P(5 6 [6\,m ~ G ]) -> 1 
exponentially fast as m — > oo. By applying Theorem [2|i) 
and (iii), we have R sum ,r,m = 0(1), which is orderwise less 
than H(mh(p m J) = @(m). (ii) If p m = l/m and f„(x m ) 
max'" i x;, then «i = fei = 0, «2 = 1> &2 = tn, and 
lim„ M0O P(S < 0)P(S > 0) = e _1 (l -e _1 ), due to TheoremSii), 



&(mh(p m )) = ©(logm). Remark: For the zero-error 



criterion, the minimum worst-case sum-rate is ©(logm) [1]. 
D. Comparison to cut-set bounds 

How do the bounds given in Sec. lIV-Bl behave in comparison 
to bounds based on cut-sets? We will show that in some cases 
they are orderwise tighter than cut-set bounds and in some 
cases they coincide with them. 

For any subset S C [l,m], let S° := [l,m] \ S. We 
can formulate a two-terminal interactive function computation 
problem with alternating message transfers [6] by regarding 
the set of source nodes in >S as supernode-*S and the other 
source nodes and the sink as supernode-»S c . The sources {X, }, e ^ 
and [Xi}j € sc are available to supernode-5 and supernode-*S c 
respectively and the function f(X m ) is to be computed at 
supernode-»S c . Let denote the directed sum-rate region 

of the two-terminal problem, which is the set of tuples 
{Rs^>S»>Rs c ->s) sucn that Rs^S' and R&^S are admissible 
directed sum-rates from >S to S° and from S c to S respec- 
tively, for two-terminal interactive function computation with 
t' alternating messages where t' < 2mr is the minimum number 
of messages needed in the two-terminal problem to simulate 
the multiround code. 

For any multiround code for a collocated network, for every 
i e [l,m], let r, denote the sum-rate of the messages broad- 
casted by node-;. This code can be mapped into a two-terminal 
interaction code for the two-terminal problem described above, 
which generates the same computation result. The directed 
sum-rate tuples is (R S ->s°,Rs°^s) = (Zi<=s r u Zte* r 0> which 
should belong to the directed sum-rate region of the two- 
terminal problem. This leads to the following cut-set bound. 

Theorem 3: (cut-set bound) For all r e Z + , 



Rsmn.r > Rem ■= mm V r;. 

V5c[l,m], (ZtssnXtsscrJeRs^ jrf 
Vi'E[l,m], n>0 



(43) 



One could also consider a different type of cut-set bound: 



max R s ""l 

SQ[l,m] ^ 



:= max 

<Sc[l,ire] 



mm 

Vfe[l,m], r ; >0 



where R™"<^ is called the bi-directional minimum sum-rate of 
the two-terminal problem given by the cut-set S. Note that 
R' cur < R cu t- In fact, R' cut can be orderwise looser than R cl „. For 
example, for the problem in Prop. Q] R cut > m and R' cul = 1 . 
Proposition 1: If X'" ~ iid Bernoulli 1 /2) and f m (x m ) = 

(0™ i x ')> then R cut > m. 

Proof: For any i e [l,m], if S — {;}, by applying the 
cut-set bound for the two-terminal interactive function com- 
putation problem [6, Corollary l(ii)], we have V (r,-, Yjfri r j) e 



Ks&, n > H{f m {X m )\{X k } keS ) = H(Xd = 1. Adding the m 
inequalities r, > 1 for all i e [1,/n], we have R cu , > m. ■ 

Since m is an admissible sum-rate for the problem stated in 
Prop.Q] the cut-set bound is tight. Note that Theorem|2ji) also 
gives the same bound R su „ ur > m. However, in the following 
case, the cut-set bound is orderwise loose. 

Proposition 2: If X m ~ iid Bernoulli 1/2) and f m (x m ) = 
min™ ! Xi , then R cut < 3m/(2 m/2 ). 

Proof: It is sufficient to show that Vz e [l,m], 
r, = 3/(2"^ 2 ) is feasible for the minimization prob- 
lem in ( 14.3b . which requires showing V*S C [1,/n], 
(ZteS^Efe^n) = (3|S|/(2'"/ 2 ),3|S c |/(2 m / 2 )) e Ks,&. Let 
Y s := (mm ie sXi(k)r k=1 ~ iid p Ys ~ Bernoulli( 1/(2^1)) and 
Y& := (mm ieS . Xiik))'^ ~ iid p Ysc ~ Bernoullia/^l)). 
The computation of the two-terminal problem can be per- 
formed by the following two schemes, (i) (One-message 
scheme) Supernode-5 sends Y$ to supernode-J?'' at the 
rate H(Y S ). Therefore (H(Y s ),0) e fts^, which implies 
that <R} := [H(Y S ),°°) x [0,oo) c '% 5 ,. (ii) (Two- 
message scheme) Supernode-»S c sends Y& to supernode-3 
at the rate H(Y^ C ). Then supernode-J? computes the sam- 
plewise minimum of Y$ and Y^<, and sends it back to 
supernode-»S c with Y& as side information available to 
both supernodes, at the rate H(min(Ys, Y&)\Ys<:)- Therefore 
(H(mm(Y s ,Y S c)\Y Sl ),H(Y Sl )) e Rs&, which implies that 
•R 2 := [H(mm(Y s ,Ysc)\Ysc),oo)x[H(Ysc),oo) c ft SSl . By 
evaluating the entropies, it can be shown that, if \S\ > m/2, 
then (Yjies r i> HieS c r i) e ^ ■> otherwise (2L & s r i> ZieS' 

n) e ft 1 . 

Therefore (2 1&s n, Zie5' n) e 

The detailed steps only for ourselves: (will be deleted in the 
final draft) If \S\ > m/2, then 

where the first inequality is because /z(p) < plog 2 (e/p) 
and the second inequality is because e < 4^. Therefore 
(ZteS^Sfes-n) = (3|<S|/(2'»/ 2 ),3|5 c |/(2"'/ 2 )) 6 If |5| < 
m/2, then H(Y$) < 3|>S c |/(2 m/2 ). If 1 < \S\ < m/2, 



1 



1 



H(min(Y s ,Y s ,)\Y Sc ) = < ^ < ^g. 

Otherwise = 0), //(min(F5, Y&W&) = = 3|5|/(2" ,/2 ). 
Therefore (2, e5 r,-, 2,^ r,) = (3|5|/(2'"/ 2 ), 3\S c \/(2>"' 2 )) e R 2 . 

■ 

Since the problem considered in Prop. [2] is a special case 
of Case 4(i), due to Theorem [2] we have R sum ,r,m - 
Therefore the exponentially vanishing cut-set bound given by 
Theorem [3] is orderwise loose. 

V. Concluding Remarks 

We studied function computation in collocated networks us- 
ing a distributed block source coding framework. We showed 
that in computing symmetric functions of binary sources, 
the sink will inevitably obtain certain additional information 
which is not part of the problem requirement. Leveraging 
this conceptual understanding we developed bounds for the 



minimum sum-rate and showed that they can be better than 
cut-set bounds by orders of magnitude. Directions for future 
work include characterizing the scaling law of the minimum 
sum-rate for large source alphabets and general multihop 
networks. 

Appendix A 
Converse proof of Theorem Q] 

Suppose a rate tuple (R\, . . .,R,) is admissible for r-round 
function computation. By Definition |2j Ve > 0, 3h(e, t), such 
that Vn > «(e), there exists an r-round distributed source 
code satisfying Vy e [1, t], {I In) log 2 \M/\ < Rj + e and 
P(Z + Z) < e. Define auxiliary random variables as follows: 
Sk e [l,n], Ui(k) := {M u X m (k-)}E and for Vi e [2,t], 
Ui := Mi. 

Information inequalities: For the first rate, we have 

n(Ri +e) > H(Mi) 

> /(X,;Mi|X™) 

= H(X X \X?) - H{X X \M U X™) 

n 

> - H(X l (k)\M l ,X'"(lc-))) 



i<=\ 



(A.l) 



For the r'-th rate, / e [2, f], let y = (/ mod m). 

n(Ri + e) > H(Mj) 

> 7(Xy;M i |M , - 1 ,X''~ 1 ,X? +1 ) 

= H(Xy|M i_1 ,X- / '" 1 ,X^ +1 ) - 7f(Xy|M i ,X- / '" 1 ,Xj +1 ) 



M'- 1 , X^ 1 , XJ +1 ) 
HiXjikWjik-lM^XJ-KXJ^)) 

-H(Xj(k)\Xj(k-),M\XJ-\XJ +l )) 

n 

> £(#(X/fc)|X m (fc-),M M ) 
*=i 

-//(X/fe)|X m (fc-), M')) 

= £/(*,(*); CWi(*), I/I -1 ). (A.2) 
*=i 

Step (a) is because the Markov chain X/A:) - (M i_1 , X m (fc-)) - 
{Xi- l {k),Xi- l {k+),X'J +x {k),X'J +x {k+)) holds for each i e [l,f] 
and A; e [l,n]. 

6 A(/t-) means (A(l), . . . ,A(k- 1)1, and A(k+) means {A(/t + 1), . . . , A(m)|. 



A = l 



Due to the condition P(Z + T) < e and the Fano's inequality 
[12], we have 

h 2 (e) + elogOZI" - 1) 

> H(Z\M') 

n 

> ^7/(Z(fc)|Z(A:-),M',X"'(A-)) 

4=1 

i ^ 77(Z(£)|M', *"'(£-)) 
£=i 

n 

= ^WPTOWiW^)- (A3) 
*=i 

Step (b) is because Z(k-) is a function of X"'(k-). 

Timesharing: We introduce a timesharing random variable 
Q taking values in [l,n] equally likely, which is independent 
of all the other random variables. For each i e [l,m], define 
Xi ■= Xi(Q), and t/, := (t/i(0,0. dATT ) becomes 

/?i + e > 



1 " 

- Y \l(Xi{k); Ui(k)) 



= 7(Xi(e);t/i(0|0 
= 7(Z 1 ((2);i7 1 (0,0 
- 7(X 1 ;f/ 1 ), (A.4) 

where step (c) is because 7(Xj(0;0 = 0, which is in turn 
implied by: (1), Q is independent of all the other random 
variables, and (2), the distribution of X\{Q) ~ px { does not 
depend on Q. Similarly, ( 1A.2I ) and (I A.31 > become 



(A.5) 



V;e[2,f], Ri + e > I(Xf,V /It/'" 1 ), 

-h 2 (e) + elog\Z\ > H(f(X m )\U% (A.6) 
n 

Concerning the Markov chains, one can verify that U\(k) — 
Xm-X^ik) and Ui - (Um, I/*" 1 ,*/*)) - (X j ~ l (k),XJ + ^(k)) 
form Markov chains for each i e [2, f], k € [1,«] and j = (i 
mod m), which imply that C/, - (C/ i_1 ,^) - forms 
a Markov chain for each / e [1, f] and y = (; mod m). 

Cardinality bounds: The cardinalities of 17, can be bounded 
as for the rate region of the two-terminal interaction problem 
[6]. But they are omitted here. 

Taking limits: As in [6], we consider a sequence {e/} which 
goes to zero as / tends to infinity. Due to the continuity of con- 
ditional mutual information and conditional entropy measures, 
all the e's in ( |A.4| )- dA6] l vanish and thus (R\, . . .,R t ) e "R r . 

Appendix B 
Proof of Lemma[3] 

For any r-round block-length n code (e', g) for comput- 
ing / without any error in a collocated network, for ev- 
ery realization m' of messages M' and for every sample 
i e [1,«], let Jl(m',i) := {x m (i)\p x „, ( j )M (x"'(i),m') > 0), 
i.e., the set of all possible z'-th source samples that are 



consistent with messages m' . We first show that Jl(m',i) is 
an /-monochromatic rectangle in (^) =1 A",, which is sim- 
ilar to the statement of Lemma Q] Due to Definition Q] 
J[{m',i) = {x m (i)\3 x m { 1 ),..., x"\i - \),x m {i- l),...,x"'(n) 
such that V/ 6 [l,?],fc = (j mod m), e/Xjt, m 7-1 ) = my}, 
where stands for (jCjt(l), . . .,x&(n)). For every A: e [1,/n], 
let S k {m', i) := \x k {i)\3 x k (Y), x k (i - 1), x k (i - 1), ... , x t (n) 
such that Vp e [0, r— l],j = k + pm,ej(x k ,m- i ) = m ; }. 
Since S k (m', i) contains all the constraints in J?[(m', ;) related to 
source node k, we have Jl(m',i) = <^)" k ' =l S k (m',i). Therefore 
J?[(m', j) is a rectangle in <^)"' =l X,. Since the code computes / 
without any error for any inputs, J[{m' , i) is /-monochromatic. 

The rest steps are parallel to the proof of Lemma [2] For all 
possible messages m', J\(m' , i) is nonempty and Sj(m', i) is ei- 
ther {0j or {lj or {0, 1}. Let 7Ci(m', /) := {z | Si(m', i) = {1}} and 
•K Q (tn',i) := {i | Si(m',i) = {0}}. Let a(m',i) := \9Ci(m',i)\ and 
/3(m', i) : = m - |7Co(m f , It can be shown that the projection 
of M(m', i) under the linear transformation 5 = (^Xi x ij given 
by M'(m',i) := ((Z£ 1 x j (0)|Pi-(ftji'W),mO > o] is an /'- 
monochromatic interval [a(m', i),fi(m', /)]. Since {[a,,, fo v ]} v ™[ is 
the collection of all the maximal /'-monochromatic intervals, 
3 v(m',i) e [l,v max ] such that W(m',i) c [a V (m<,i) A(m',i)]- 
Therefore (i) holds. Since = [a(m', 0,jS(m', /)] £ 

[av(m',i>^v(m',o]' we have a(m',i) > fl v (m',0 andy6(m',0 < fc v (m',0- 
Therefore (ii) holds. 

Appendix C 
Proof of Theorem [2] 

Proof of Theorem [2] ( i): We first state a lemma. 

Lemma 4: If F* € {0, 1}* is a random vector and Sv - 
F, then ff(F*) < kh(E(S Y )/k). 
Proof: 

H(Y«) < J] //(F) = J /,(E(F,)) § ftft (^S) = ^ (Mz)) . 

i= 1 i= 1 

Step (a) is due to the concavity of h(x) and the Jensen's 
inequality. ■ 
Define an auxiliary random variable V by V := v if and only 
if S G [a v ,£> v ]. Lemma|2ji) implies that pv\u<{ v W) = <S v ,v(k<) an d 
thus V = v(U'). Due to Corollary Q] we have 

R sum , r = min /(*'"; t/') 

= mm[H(X"') - H(X"'\U\ V)] 

v=l 

where U' are subject to all the Markov chain and conditional 
entropy constraints in 13.11 . Due to Lemma EJii), given that 
U' = u< e supp(/?£/<), Vi e 7Ci(m'), ^, = 1, and Vi' e Kq{u'), 
Xy = 0. Therefore there are at most (m - \K\(u r ) + TCo(m')I) = 
(b V ( U <) - a V ( U ty) undetermined sources. When a V („t) = fo v («')' a ^ 
sources are determined. Therefore, for those v satisfying a,, = 



b v , H(X m \U t , V = v) = holds. For other v's we have 

H(X"'\U', V = v) 

- 2ff(jr|tf' = i^V=v)j^,y(ii'|v) 



< (Z> v -a v ) J^(e( 



5 - a, 



(c) 



5 - a v 



C/ t = M',y=vjj/J £/ ,|v(«'lv) 

V = v)|, (C.8) 



where the summations are through all u' e supp(put\v(-\v)). 
Step (b) is due to Lemma [4] Step (c) is due to the concavity 
of h{x) and the Jensen's inequality. Combining ( IC.7l i and ( IC.8I ) 
leads to the statement of Theorem |2i). 
Proof of Theorem [2] ( ii): 

Lemma 5: Let S be a random variable taking integer values. 
Let integers a\,b\,a2,b2 satisfy a\ < b\,ai < bn and «2 = 
b\ + 1. Then 



J {b v -a v )h{ 



E(S\Se[a v ,b v ])-ay 
b v —a v 



)P(5 € [a v A,]) 



< (fca- ai )A( gglg|^=a- )p(S e [a u b 2 ]). (C.9) 
Proof: If a i +b\ and «2 # ^2, it is sufficient to show that 



bi-b\ i-/rt\ P(5e[ai,fci]) , fl 2-«i /./i \t 
b,-a, nyU 'P(Se[a,,b 2 ]) b 2 -a, " K L ' V(S £[a,,b 2 ]) 



,P(Se[£2>]) 



Z b r -a r h l B{.S\Se[a v M)-ciA 



F(5e[a,.,fc,.]) 
P(S£[o 1( 6 2 ]) 



r=l,a,.*6, 



< (62-ai)A( 



E(5|5E[ai,fc 2 ])-ai 



)P(Xe[«i,W, 



which is guaranteed by the concavity of and the Jensen's 
inequality. If a\ — b\ or a 2 — bi, drop the corresponding term 
in the summation, and the proof continues to hold. ■ 
The left hand side of 1C.9I I contains the terms in the lower 
bound in Theorem|2ji) that correspond to intervals [a\, bi\ and 
[«2,^2]- The right hand side of lC.9t is a term corresponding 
to [a 1,^2]. Lemma |5] shows that the lower bound given in 
Theorem [2ji) will decrease when we combine two adjacent 
intervals. 

In order to prove Theorem [2] (ii), it is sufficient to show 
that R SU m,r ^ mh(p)P(S < b r )F(S > b v ) holds for every v e 
[L(vmax-l)]- For any v e [1, v max -l], let b := b v , p := P(5 < 
b) and p\ := 1 - po. If b ± and b + m-\, apply Lemma 
to Theorem [2ji) by combining all the intervals greater than 
b into [b + l,m] and all the intervals not greater than b into 
[0, b], we have 

R sum , r > mh( P )-b Po h (*m&)-( m -b-l) Pl h { ^TbY' 1 ) • 

(CIO) 

If & = or & = m - 1 , drop the second or the third term on 
the right side of ( IC.lOl l and the corresponding terms in the 
following steps, and the proof continue to work. In order to 
show that the right side of (IC.lOl l is not less than mh(p)popi, 
it is sufficient to show that 

Aih{ai) + A 2 h(a 2 ) + A 3 h(0) + A 4 h(l) < h(p), (C.ll) 



where 



bpo 



m(l -popi) 
E(5|5 <b) 



*2 



(m-b - l)pi 



U2 



m(l -p p\) 
E(S\S >b+ l)-b-l 



b m - b - 1 

and /l3,/?4 are arbitrary real numbers. (IC- lib is guaranteed by 
the Jensen's inequality if A\a\ + Aiaj + M — p, Ya=\ = 1 
and Ai > 0, i = 1 , . . . , 4. The first two conditions imply that 

!b+ 1 



po(l -p) m-b 
A 3 = | pt\, A4 = 



PP\ 



-Pa 



\-p pi\m-mp j l-poPi\mp 

We need to verify that A3 > and A4 > 0. In order to get 
A4 > 0, it is sufficient to verify that 

b + 1 



po = ¥(S <b)< 



mp 



(C.12) 



When b > mp-l, (1C.12I > holds immediately. When b < mp—1, 
we verify ( 1C.121 I as follows. 

The probability mass function ps(s) is nondecreasing when 
s < fis, where ps = \. m P + p\ is the mode of the binomial ran- 
dom variable S . Therefore {P(S < b)}^"^ is a convex sequence 
[13], which implies that for all integers < b < \_mp\, the point 
(b, P(5 < b)) is below or on the line segment joining the point 
(0, P(5 < 0)) = (0, (1 -p) m ) and the point (\mp\, P(5 < [mp])). 
Since 



mp(\ 



p) m < mp(l - pf'- 1 < g - p)"'- 1 = 1 



we have (1 - p) m < l/mp. Also, P(5 < Imp}) < 1 < 
{\mp\ + l)/mp. Therefore the line segment joining (0, (1 -p) m ) 
and (\mp\,V(S < \mp\)) is below the line segment joining 
(0, l/mp) and {\mp\, {\mp\ + l)/mp), which is the graph of 
the function (b + l)/mp when < b < [mpj. Therefore we 
have shown that (IC. 12b holds for b < [mp] and completed the 
proof for A4 > 0. Similarly we have /I3 > 0. 
Proof of Theorem^} (Hi): Let r — 1 and for each ; e [l,m], 
define 



0, if 3 v, s.t. NiiU*- 1 ) > a v ,N {U 1 - 1 ) > m - b v , 
Xi, otherwise, 



where N x {U'~ l ) is the number of times the symbol x occurs 
in the sequence U'~ l . Define a random variable V by V := v 
if and only if S e [a v ,b v ]. Define a random variable K by 
K := min{/|/ E [l,m], iVi(t/ ! ') > ay^oiU') > m - b v ). In 
other words, we define U\ = X\, U2 = X2, and so on, until 
after K steps, U K = X K contains at least a v ones and at least 
(m - b v ) zeros, so that for arbitrary values of the remaining 



zeros have appeared. Once it happens, the sum of sources 
definitely falls into an /'-monochromatic interval [a v ,b v ] so 
that the desired function is determined. Thus the computation 
for this sample is stopped. K is the stopping time of sending 
data. 

For any source sequence x"\ the corresponding values k and 
u m satisfy u k = x k and m™ +1 = 0. Therefore 

Pu>»(u"') = p u *(u k ) Pu ,j Ut (0\u k ) = Px *(x k ). 



Then we have 



Roun.1 < KX"';U m ) 
= H(U m ) 



E log 



= E log 



1 



Pu™(U m ) 
1 



= E(H(X K \K)) 
= B(K)h(p) 



■max 

= h(p)Y J Pv(v)B(K\V = v). (C.13) 



v=l 



We need to bound E(AjV = v) with respect to the joint 
distribution of (K, V), which is given by: if k e [l,m],v € 
[1, v m ax], PKv(k, v) = P(X k contain a v ones and (m - b v ) zeros, 
but X k ~ l do not ); otherwise picvik, v) = 0. 

For each v, we can define another random variable K' v as 
the number of iid Bernoulli(p) trials to get a v ones and (m - 
b v ) zeros. In other words, pK[{k) = P(Y k contain a v ones and 
(m - b v ) zeros, but Y k ~ l do not ), where T, ~ iid Bernoulli(/j) 
for i e N. Note that unlike K, which does not exceed m, K' v 
could be arbitrarily large. 

Since Vv e [l,v max ],V£ e [l,m\,p K {k) = p KV (k,v), the 
conditional distributions of (K\V = v) and {K[\K[, < m) are the 
same. Therefore 



E{K\V = v) = E(K[\K' V < m) < E(K' V ). 



(C14) 



The last step is because for any random variable X and Va e R, 
E(X\X <a)< E(X). 

Then, define two independent random variables W Vr i and 
W v ,o as follows: is the number of iid Bernoulli(/?) trials to 
get a Y ones. Wv,o is the number of iid Bernoulli(/5) trials to get 
(m - b x ) zeros. They are negative binomial distributed random 
variables and E(W v> i) = a v /p and E(W Vj o) = (m - b v )/(l - p). 



sources x"' ., S definitely belongs to [a v ,b v ], which means Since if Y W, ' A contains a v ones and Y, 



that the desired function is determined. After the K-th step, 
no information is sent, because C/S +] = 0. One can verify 
that U' satisfy the Markov chains and the conditional entropy 
equality in d3.U . Intuitively speaking, the above definition of 
U' corresponds to the following one-round coding scheme: For 
each sample, the source nodes keep sending the original data 
until there exists v € [1, v max ] such that a v ones and (m - b v ) 



contains (m - b v ) 



zeros, then Y Wv - i+WrS contains at least a v ones and (m - b v ) 
zeros, we have K' v < (W Vs \ + W Vi o), which implies that 

E(K[) < E(W v ,i) + E(W,, ) = a v /p + (m- b v )/(l - p). (C.15) 



Combining (1C.13I I. ( IC. 14b and ( IC. 15b leads to the statement 
of Theorem HJiii). 



References 



[1] A. Giridhar and P. Kumar, "Computing and communicating functions 
over sensor networks," IEEE Journal on Selected Areas of Communica- 
tion, vol. 23, no. 4, pp. 755-764, Apr 2005. 

[2] S. Subramanian, P. Gupta, and S. Shakkottai, "Scaling bounds for func- 
tion computation over large networks," Proc. IEEE Inttl. Symp. Info. The- 
ory (ISIT), Jul. 2007. 

[3] R. Appuswami, M. Franceschetti, N. Karamchandani, and K. Zeger, 
"Network coding for computing," Proc. AUerton Conference, 2008. 

[4] V. Prabhakaran, K. Ramchandran, and D. Tse, "On the role of interaction 
between sensors in the CEO problem," Proc. AUerton Conference, 2004. 

[5] A. Orlitsky and J. R. Roche, "Coding for computing," IEEE 
Trans. Info. Theory, vol. JT-47, no. 3, pp. 903-917, Mar 2001. 

[6] N. Ma and P. Ishwar, "Two-terminal distributed source coding with al- 
ternating messages for function computation," Proc. IEEE International 
Symposium on Information Theory (ISIT), 2008. 

[7] R. Gallager, "Finding parity in a simple broadcast network," IEEE 
Tran. Info. Theory, vol. IT-34, pp. 176-180, March 1988. 

[8] L. Ying, R. Srikant, and G. Dullerud, "Distributed symmetric function 
computation in noisy wireless sensor networks," IEEE Tran. Info. The- 
ory, vol. IT-53, pp. 4826^1833, Dec. 2007. 

[9] O. Ayaso, D. Shah, and M. Dahleh, "Information theoretic bounds on 
distributed computatiton," submitted to IEEE Transaction on Information 
Theory, 2008. 

[10] B. Nazer and M. Gastpar, "Computation over multiple-access channels," 
IEEE Tran. Info. Theory, vol. IT-53, no. 10, pp. 3498-3516, Oct 2007. 

[11] E. Kushilevitz and N. Nisan, Communication Complexity. Cambridge 
University Press, 1997. 

[12] T. M. Cover and J. A. Thomas, Elements of Information Theory. New 
York: Wiley, 1991. 

[13] J. E. Pecaric, F. Proschan, and Y. L. Tong, Convex Functions, Partial 
Orderings, and Statistical Applications. Academic Press Inc, 1992. 



